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Abstract 

Consider  the  following  supposedly-simple  problem: 

compute  x  satisfying  x  £  S  , 

where  S'  is  a  convex  set  conveyed  by  a  separation  oracle,  with  no  further  information  (e.g., 
no  bounding  ball  containing  or  intersecting  S,  etc.).  Our  interest  in  this  problem  stems  from 
fundamental  issues  involving  the  interplay  of  (i)  the  computational  complexity  of  computing  a 
point  x  £  S,  (ii)  the  geometry  of  S,  and  (iii)  the  stability  or  conditioning  of  S  under  perturbation. 
Under  suitable  definitions  of  these  terms,  we  show  herein  that  problem  instances  with  favorable 
geometry  have  favorable  computational  complexity,  validating  conventional  wisdom.  We  also 
show  a  converse  of  this  implication,  by  showing  that  there  exist  problem  instances  in  certain 
families  characterized  by  unfavorable  geometry,  that  require  more  computational  effort  to  solve. 
This  in  turn  leads,  under  certain  assumptions,  to  a  form  of  equivalence  among  computational 
complexity,  the  geometry  of  S,  and  the  conditioning  of  S.  Our  measures  of  the  geometry  of 
S,  relative  to  a  given  (reference)  point  x,  are  the  aspect  ratio  A  =  R/r,  as  well  as  R  and  1/r, 
where  B(x,R)  D  S  contains  a  ball  of  radius  r.  The  aspect  ratio  arises  in  the  analyses  of  many 
algorithms  for  convex  problems,  and  its  importance  in  convex  algorithm  analysis  has  been  well- 
known  for  several  decades.  However,  the  terms  R  and  1/r  in  our  complexity  results  are  a  bit 
counter-intuitive;  nevertheless,  we  show  that  the  computational  complexity  must  involve  these 
terms  in  addition  to  the  aspect  ratio  even  when  the  aspect  ratio  itself  is  small.  This  lower-bound 
complexity  analysis  relies  on  simple  features  of  the  separation  oracle  model  of  conveying  S;  if 
we  instead  assume  that  S  is  conveyed  by  a  self-concordant  barrier  function,  then  it  is  an  open 
challenge  to  prove  such  complexity  lower-bound. 


‘This  research  has  been  partially  supported  through  the  Singapore-MIT  Alliance  and  AFOSR  Grant  FA9550-08- 
1-0350 

^MIT  Sloan  School  of  Management,  50  Memorial  Drive,  Cambridge,  MA  02142,  USA,  email:  rfreund@mit.edu 
^Dcpt.  de  Ingeniena  Industrial  y  Sistemas,  Facultad  de  Ingenieria,  Pontificia  Universidad  Catolica  de  Chile, 
Campus  San  Joaquin,  Vicuna  Mackenna  4860  Santiago,  CHILE,  email:  jvera@ing.puc. cl 
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1  Introduction,  Motivation,  and  Discussion 

Consider  the  following  supposedly-simple  problem: 

compute  x  satisfying  x  G  S  ,  (1) 

where  S  C  X  is  a  convex  set  (bounded  or  not,  open  or  not)  conveyed  by  a  separation  oracle  with  no 
further  information  (e.g.,  no  bounding  ball  containing  or  intersecting  S,  etc.),  and  X  is  a  (finite)  n- 
dinrensional  vector  space.  Our  interest  in  (1)  stems  from  fundamental  issues  involving  the  interplay 
of  three  notions:  (i)  the  computational  complexity  of  computing  a  point  x  G  S,  (ii)  the  geometry 
of  S,  and  (iii)  the  stability  or  conditioning  of  S.  In  this  paper  we  focus  on  the  equivalence  of 
computational  complexity  and  a  suitable  measure  of  the  geometry  of  S,  which  leads  under  certain 
assumptions  to  an  equivalence  of  all  three  notions. 

There  are  two  standard  information  models  for  convex  sets,  the  separation  oracle  model  and 
the  (self-concordant)  barrier  model.  A  separation  oracle  for  S.  see  [12],  is  a  subroutine  that,  given 
a  point  x  as  input,  returns  the  statement  “x  G  S"  if  indeed  this  is  the  case,  or  if  x  ^  S,  returns  a 
hyperplane  H  with  the  property  that  x  G  H~  and  S  C  H++ .  Here  H  C  denotes  a  hyperplane, 
and  H~  and  H+  ~  and  H++ )  denote  the  two  closed  (open)  halfspaces  of  3in  bounded  by 
H.  From  a  computational  viewpoint,  the  information  about  H+  is  described  by  a  nonzero  vector 
h  G  $tn  and  scalar  a  for  which  H+  =  {x  G  3in  :  hT x  >  a}.  For  any  separation  oracle  based 
algorithm,  we  define  the  number  of  iterations  of  the  algorithm  to  be  the  number  of  oracle  calls,  i.e. , 
we  use  oracle  calls  and  iterations  interchangeably. 

The  separation  oracle  model  applies  readily  to  the  case  when  S  is  the  feasible  region  of  a  conic 
linear  system: 

5  :=  {x  G  :  b  -  Ax  G  K}  ,  (2) 

where  A  is  a  linear  operator  from  A  to  a  (finite)  m-dimensional  vector  space  Y,  b  G  Y,  K  C  Y  is 
a  closed  convex  cone,  and  d  =  (A,  b )  is  the  data  for  the  system.  In  this  case  a  separation  oracle  for 
the  cone  K  can  be  used  to  easily  construct  a  separation  oracle  for  S. 


1.1  Discussion  of  Main  Results 

A  representative  separation  oracle  algorithm  for  solving  (1)  is  the  ellipsoid  algorithm  [12],  see  also 
[6].  The  ellipsoid  algorithm  is  easy  to  implement  and  has  very  good  theoretical  complexity.  We 
assume  the  reader  is  familiar  with  the  concepts  underlying  this  algorithm.  In  consideration  of  this, 
suppose  that  we  know  a  priori  a  vector  x  and  a  positive  scalar  R  with  the  property  that  B(x,  R)nS 
has  positive  volume  (where  B(c,  p )  denotes  the  ball  of  radius  p  centered  at  c).  This  last  assumption 
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can  also  be  stated  as  saying  that  there  exist  values  R  and  r  >  0  for  which  it  holds  that: 

B(x,  R)  n  S  contains  a  ball  of  radius  r  .  (3) 

Then  the  ellipsoid  algorithm  for  solving  (1)  can  be  initiated  with  the  ball  B(x,  R),  and  the  number 
of  oracle  calls  (equivalently,  the  number  of  iterations)  of  the  algorithm  required  to  solve  problem 
(1)  is  known  (see  [12])  to  be  at  most 


We  will  in  general  refer  to  a  ratio  of  the  form  A  :=  R/r  where  R,  r  satisfy  (3)  as  an  aspect  ratio  of 
S  relative  to  the  reference  point  x.  The  aspect  ratio  arises  in  the  analysis  of  many  algorithms  for 
convex  problems,  and  its  importance  in  convex  algorithm  analysis  dates  back  at  least  four  decades, 
see  e.g.,  Rosenblatt’s  1962  book  [18]. 

The  ellipsoid  algorithm  requires  prior  specification  of  x  and  R  to  determine  the  starting  ball 
(ellipsoid)  with  which  to  start  the  method;  the  condition  that  (3)  holds  for  some  r  >  0  is  also 
needed  in  order  to  guarantee  convergence.  However,  at  least  in  theory  as  well  as  in  the  case  of 
typical  instances  of  (1)  when  S  is  in  the  conic  format  (2),  a  priori  values  of  x  and  R  satisfying 
(3)  are  either  not  known  or  involve  excessively  conservative  (large)  values  of  R  »  0  for  special 
structure  and  data  representation  of  S.  For  example,  suppose  S  =  [x  £  Sn  :  Ax  <  b}  for  known 
rational  data  d  =  (. A ,  b )  whose  bit-encoding  is  of  length  L.  Then  one  can  pre-determine  values 
of  x  :=  0  (e  Uin),  R  :=  2nL,  and  r  :=  2~nL  for  which  B(x,R)  n  S  contains  a  ball  of  radius  r  if 
and  only  if  (1)  has  an  interior  solution.  This  observation  underlies  Khachiyan’s  proof  that  linear 
optimization  is  polynomial-time  in  the  bit-model  in  [7]  (see  also  [2]  and  [6]);  however  in  most 
instances  there  are  significantly  better  values  of  R  and  r  than  those  above.  As  another  example, 
suppose  S  =  {x  £  J?n  :  Q\X\  + . .  .  +  Qnxn  h  Qo}  where  Q,  G  Srn  are  integral  symmetric  matrices  of 
order  m,  for  %  =  0, . . . ,  n.  Porkolab  and  Khachiyan  [9]  show  that  if  S  is  bounded  then  S  C  B( 0,  R) 

for  R  =  e(Lm0,'""'(n’rn  w}iere  £  js  thg  maximum  bit-length  of  the  coefficients  of  the  matrices 
Qo, ,  Qn .  While  this  bound  on  R  is  pre-determined,  it  is  doubly  exponential  in  at  least  one  of 
the  dimensions  of  the  problem.  In  Section  2  we  present  an  extension  of  the  ellipsoid  algorithm  that 
eliminates  the  need  for  any  a  priori  information  about  R  and/or  r,  and  we  prove  an  upper  bound 
on  the  number  of  iterations  required  to  solve  (1)  using  this  extension  of  the  ellipsoid  algorithm, 
given  by: 

2(n  +  1)  Q  ln(ra)  +  1.16  +  rain  Q  +  ^  +  ln(l  +  R)j  , 

where  R,  r  are  any  existential  values  satisfying  (3)  but  are  not  required  to  be  known  to  run  the 
algorithm  (see  Theorem  2.2).  Notice  that  this  iteration  bound  involves  not  just  the  aspect  ratio 
R/r  but  also  involves  both  1/r  and  R  itself.  The  complexity  bound  essentially  states  that  sets  S 
with  favorable  geometry  relative  to  x,  in  the  sense  that  none  of  the  quantities  R/r ,  R,  or  1/r  are 
very  large,  will  not  require  too  much  computation  to  solve. 
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The  presence  of  the  aspect  ratio  term  A  =  R/r  in  the  above  complexity  bound  is  to  be  expected, 
especially  in  light  of  its  presence  in  the  case  when  R  is  known  and  given  to  the  algorithm  a  priori. 
This  still  begs  the  question  of  whether  the  terms  R  and  1/r  must  be  present  or  whether  they  can  be 
removed  from  the  complexity  upper  bound  using  a  different  algorithm  and/or  a  different  analysis. 
Notice  that  even  if  R/r  is  small,  say  at  most  an  absolute  constant,  the  values  of  R  and/or  1/r  might 
still  be  very  large,  yielding  perhaps  an  overly  conservative  iteration  complexity  upper  bound,  as 
shown  in  the  following  two  examples. 


Example  1.1  Let  n  =  1,  x  =  0,  and  S  =  [10  6,3  x  10  6].  Then  r  =  10  6,  R  =  3  x  10  6  yields 
R/r  =  3  but  1/r  =  106. 


Example  1.2  Let  n  =  1,  x  =  0,  and  S  =  [106,  3  x  106].  Then  r  =  106,  R  =  3  x  106  yields  R/r  =  3 
butR  =  3  x  106. 

Queried  a  different  way,  are  all  three  geometric  measures  R/r ,  R,  and  1/r  necessary  components 
of  the  computational  complexity  of  solving  (1)?  We  resolve  this  question  in  Section  3,  where  we  show 
that  the  dependence  of  the  complexity  bound  on  R  and  1/r  (as  well  as  on  R/r )  cannot  be  removed. 
In  Theorems  3.1,  3.2,  and  3.3,  we  show  under  suitable  assumptions  for  some  specific  families  of 
problem  instances,  that  any  separation  oracle  algorithm  for  solving  (1)  must,  for  some  instance 
in  the  requisite  family,  require  at  least  [log2(.R/r)  —  lj ,  or  [k)g2  log2(i?  +  1)J,  or  |_log2  log2(l/4r)J 
iterations,  respectively.  While  these  lower  bounds  are  not  of  the  same  order  as  the  upper  bound 
presented  above,  they  do  involve  the  same  three  geometric  quantities  R/r ,  R,  and  1/r. 

Taken  together,  these  results  demonstrate  a  certain  equivalence  between  computational  com¬ 
plexity  and  problem  instance  geometry  of  S  as  measured  by  R/r,  R,  and  1/r.  Indeed,  while 
problems  with  favorable  geometry  do  not  require  too  much  computational  effort  to  solve,  there 
exist  problem  instances  in  certain  families  with  unfavorable  geometry,  that  require  more  computa¬ 
tional  effort  to  solve.  This  equivalence  ties  in  nicely  with  results  regarding  the  interplay  of  stability 
and  conditioning,  problem  geometry,  and  computational  complexity  for  problems  in  conic  format 
(2).  Considering  S  defined  by  (2)  for  data  d  =  ( A,b )  and  keeping  the  cone  K  fixed,  we  measure 
the  condition  number  of  (2)  using  the  “distance  to  infeasibility”  which  we  now  briefly  review.  Let 
L(X,  Y )  denote  the  space  of  linear  operators  from  X  to  Y  and  let  XI  C  L(X,  Y)  x  Y  denote  those 
data  pairs  d  =  ( A,b )  for  which  S  given  by  (2)  is  nonempty.  For  d  =  ( A,b )  £  M.,  let  p(d)  denote 
the  “distance  to  infeasibility”  for  (2),  namely: 

P(d )  :=  min  {||Ad||  :  d  +  Ad  (/  M}  , 

Ad=(AA,Ab) 

under  suitably  defined  norms  on  spaces  and  operators,  see  [14].  Then  p(d)  denotes  the  smallest 
perturbation  of  our  given  data  d  which  would  render  the  resulting  system  in  (2)  infeasible.  Next 
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let  C(d)  denote  the  condition  measure  of  (2),  namely  C(d)  =  ||d||//o(d),  which  is  a  scale-invariant 
reciprocal  of  the  distance  to  infeasibility.  There  are  strong  connections  between  C(d)  and  bounds  on 
the  stability  of  S  under  data  perturbation,  see  Renegar  [14].  It  is  shown  in  Renegar  [15]  and  in  [3] 
that  problems  with  favorable  condition  numbers  C(d)  do  not  require  too  much  computational  effort 
to  solve  using  interior-point  methods  and  the  ellipsoid  method,  respectively.  Also,  using  x  =  0,  it  is 
shown  in  [4]  that  a  favorable  value  of  C(d)  implies  favorable  geometry  of  S,  namely  favorable  values 
of  R/r,  R,  and  1/r,  and  that  a  converse  of  this  statement  holds  under  an  assumption  of  “conic 
curvature”  defined  and  shown  in  [1].  Taken  together,  these  cited  results  in  combination  with  the 
results  developed  in  this  paper  demonstrate  an  equivalence  between  favorable  geometry,  favorable 
complexity,  and  favorable  conditioning  under  suitable  assumptions. 

Last  of  all,  we  remark  that  we  have  only  shown  an  equivalence  between  the  geometry  of  S  and 
computational  complexity  in  the  separation  oracle  model,  and  not  in  the  (self-concordant)  barrier 
model.  It  is  shown  in  [5]  that  favorable  geometry  implies  favorable  computational  complexity  of 
an  interior-point  method.  However,  it  is  an  open  challenge  to  prove  an  assertion  that  unfavorable 
geometry  implies  (say,  in  the  worst  case)  a  large  number  of  iterations  of  an  interior-point  method. 

The  rest  of  the  paper  is  organized  as  follows.  In  Sections  2  and  3  we  present  our  lower  and 
upper  bounds  on  the  complexity  of  solving  (1),  respectively.  Section  4  discusses  possible  extensions 
and/or  strengthening  of  the  complexity  bounds. 

Notation.  We  assume  that  X  is  equipped  with  an  inner-product  norm  ||u||  :=  \J ( v ,  v ).  For 
convenience  we  identify  X  with  iRn  and  the  inner  product  (•,•}  with  the  standard  scalar  prod¬ 
uct  ( v,w )  =  vTw  =  J2j=ivjwj-  Let  Sk ,  S+,  and  S++  denote  the  set  of  symmetric  matrices, 
symmetric  positive  semi  definite  matrices,  and  symmetric  positive  definite  matrices  of  order  k , 
respectively.  Let  “y”  denote  the  Lowner  partial  ordering  on  symmetric  matrices:  A  A  B  if 
and  only  if  A  —  B  is  positive  semidefinite.  Let  Qk  denote  the  /c-dimensional  second-order  cone 
{x  G  :  || (x2,  ■  ■  •  ,Xfc)||2  <  x\}  for  k  >  2. 


2  Upper  Bounds  on  Complexity  of  (1)  via  an  Extension  of  the 
Ellipsoid  Algorithm 


We  first  review  some  basic  results  about  the  ellipsoid  algorithm,  see  Nemirovsky  and  Yudin  [12], 
also  Grotschel  et  al.  [6],  and  then  present  an  extension  of  the  ellipsoid  algorithm  that  solves  (1)  in 
the  absence  of  any  bounding  information  about  S.  The  ellipsoid  method  applied  to  solve  (1)  for  a 
given  convex  set  S  C  is  completely  specified  by  the  separation  oracle  for  S  and  the  center  x° 
and  shape  matrix  Qq  y  0  of  the  starting  ellipsoid  E°  :=  {x  G  :  (x  —  x°)TQo{x  —  x°)  <  1}.  By 
a  simple  change  of  variables,  there  is  no  loss  of  generality  in  assuming  that  Q o  =  ( p°)~2I  for  some 
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p°  >  0,  whereby  E°  =  B(x° ,  p°)  and  the  information  content  of  the  starting  ellipsoid  is  simply  the 
center  x°  and  radius  p°  of  the  starting  ball. 

Suppose  that  we  know  a  priori  a  vector  x  and  a  positive  scalar  R  with  the  property  that 
B(x,  R)  n  S  has  positive  volume.  Then  the  ellipsoid  algorithm  for  solving  (1)  can  be  initiated  with 
the  ball  B(x,  R).  The  following  is  a  generic  result  about  the  performance  of  the  ellipsoid  algorithm, 
where  in  the  statement  of  the  theorem  “vol(T)”  denotes  the  volume  of  a  set  T : 


Theorem  2.1  Ellipsoid  Algorithm  Theorem  with  known  R,  from  [12],  also  [6].  Suppose 
that  a  vector  x  and  a  positive  scalar  R  are  known  with  the  property  that  the  set  B(x,R)  n  S  has 
positive  volume.  Then  if  the  ellipsoid  algorithm  is  initiated  with  the  Euclidean  ball  B(x,R),  the 
algorithm  will  compute  a  solution  of  (1)  in  at  most 

\2nln 

Vvol  (B(x,R)rS)J 

iterations,  where  each  iteration  makes  one  call  of  the  separation  oracle  for  S. 


It  is  often  convenient  to  restate  this  result  using  radii  of  certain  balls  rather  than  volumes  of 
certain  sets.  The  supposition  of  positive  volume  in  this  theorem  is  equivalent  to  the  condition  that 
there  exist  values  R  and  r  >  0  that  satisfy  (3).  Then  one  then  obtains,  for  example: 


Corollary  2.1  (see  [12])  Suppose  that  S  is  given  via  a  separation  oracle,  and  that  the  ellipsoid 
algorithm  is  applied  to  solve  (1)  starting  with  E°  :=  B(x,R)  for  some  given  x  and  R.  If  S,  R,  and 
r  satisfy  (3)  for  some  r  >  0,  then  the  ellipsoid  method  will  solve  (1)  in  at  most 


2 n2  In 


iterations,  where  each  iteration  makes  one  call  of  the  separation  oracle. 


Proof:  Let  v(n)  denote  the  volume  of  the  n-dimensional  unit  ball,  namely: 

W2 

”(")  =  rw^TT)  • 

see  [6].  Letting  B(y,  r)  denote  the  ball  of  radius  r  contained  in  B(x,  R)  fl  S,  we  have 

vol  (B(x,R))  vol  {B(x,R))  =  ( v(n)Rn\  = 

vol(B(x,  R)  (1  S)  ~  vol (B(y,r))  \v(ri)rn  )  ’ 


(4) 
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and  the  result  follows  using  Theorem  2.1  .| 

Note  that  the  ellipsoid  algorithm  requires  prior  specification  of  x  and  R  just  to  implement 
the  method;  the  condition  that  (3)  holds  for  some  r  >  0  is  also  needed  in  order  to  guarantee 
convergence.  Of  course,  a  priori  values  of  x  and  R  for  which  (3)  is  true  (for  some  r  >  0)  are 
typically  either  not  known  or  involve  excessively  conservative  (large)  values  of  R  »  0  for  special 
structure  and  data  representation  of  S  (see  the  discussion  in  Section  1.1).  Except  for  instances  such 
as  these  where  prior  information  about  special  structure  and  data  for  S  is  given,  such  bounds  on  R 
are  not  known.  Despite  this  lack  of  prior  information  in  general,  one  can  still  utilize  the  ellipsoid 
algorithm  for  solving  (1)  by  using  a  standard  “lift  and  conify”  transformation,  see  [3],  that  we  now 
review  and  extend. 

For  any  given  value  of  x  (think  x  =  0  for  convenience),  define: 

Wx  :=  {(w,  9)  G  sRn+l  :  w  -  x  G  9{S  -  x),  9  >  0}  .  (5) 

Notice  that  Wx  (~l  {(re,  9)  :  9  =  1}  =  S  x  {1},  i.e.,  the  restriction  of  Wx  to  the  slice  of  (x,  9)  G  Kn+1 
defined  by  9  =  1  is  simply  S.  Also,  Wx  is  a  (translated)  convex  cone  in  3f£n+1  with  base  (x,0). 
Therefore  Wx  is  constructed  by  first  lifting  S'  to  S'  x  {1}  c  and  then  conically  extending 

S  x  {1}  using  the  base  point  (x,  0),  hence  the  term  “lift  and  conify.”  One  can  solve  (1)  by  instead 
working  with  the  following  equivalent  problem  in  one  higher  dimension: 

compute  (w,  9)  satisfying  (w,  9)  G  Wx  .  (6) 

The  equivalence  of  (1)  and  (6)  follows  since  solutions  of  one  system  can  be  converted  to  solutions 
of  the  other  system  as  follows: 

(w,9)eWx  =^>  x  :=  x  +  (w  —  x)/6  G  S 

x  G  S  =£•  (w,  9)  :=  (a(x  —  x)  +  x,  a)  G  Wx  for  all  a  >  0  . 

Furthermore,  a  separation  oracle  for  S  can  be  readily  converted  to  a  separation  oracle  for  Wx,  as 
follows.  If  ( w,9 )  is  a  given  point,  first  check  if  9  >  0;  if  not,  then  H+  :=  {( w,9 )  :  9  >  0}  is  the 

requisite  separating  halfspace.  If  9  >  0,  next  check  if  x  :=  x  +  (w  —  x)/9  is  in  S.  If  so,  then 

(w,  9)  G  Wx  and  we  are  done.  Otherwise,  the  separation  oracle  for  S  outputs  ft  /  0  for  which 
hT x  >  hT x  for  all  x  G  5,  which  then  implies  that  hT  (x  +  (w  —  x)/6)  >  hT  (x  +  (w  —  x)/9^j  for  all 

(w,6)  G  Wx.  Simplifying  yields  H+  :=  {(w,6)  :  9hTw  —  9hrx  >  9{hTw  —  hTx)}  as  the  requisite 
separating  halfspace  for  (w,  9)  in  this  case. 

Because  Wx  is  a  (translated)  convex  cone  in  Kn+1  with  base  (x,0),  Wx  contains  points  in  the 
(n  +  l)-dimensional  unit  ball  centered  at  (x, 0),  which  we  denote  by  Bx+1: 

Bx+1  :=  Bn+  i((x,0),  1)  :=  |(to,0)  G  Kn+1  |  yj {w  -  x)T{w  -  x)  +  92  <  ij  . 
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Therefore,  given  only  x,  one  can  apply  the  ellipsoid  algorithm  to  solve  (6)  (and  hence  solve  (1)) 
using  the  initial  ball  Bx+1,  yielding  the  following  “extended”  version  of  the  basic  ellipsoid  algorithm: 


Extended  Ellipsoid  Method  for  Solving  (1)  with  Unknown  R 

(a)  Input:  separation  oracle  for  S,  and  initiating  point  igS". 

(b)  If  x  £  S,  output  x  and  Stop.  Otherwise  construct  Wx  using  (5)  and  run  the  ellipsoid 

algorithm  in  Kn+1  starting  with  Bx+1  to  compute  a  point  in  Wx.  Output  (■ w,9 )  £  Wx. 

(c)  Set  x  «—  x  +  (w  —  x)/6  and  Stop. 


In  order  to  prove  a  complexity  bound  using  the  above  extension  of  the  ellipsoid  method,  we 
must  bound  the  ratio  of  the  volume  of  Bx+1  to  the  volume  of  Bx+ 1  0  Wx.  This  is  accomplished  in 
the  following  lemma,  a  variant  of  which  was  first  presented  in  [3] : 


Lemma  2.1  Suppose  that  S  O  B(x,R )  contains  a  ball  of  radius  r  >  0.  Then 
vol  (B* 


In 


J°1  (Bn+i  n  w 


I'nwf  )  ~  l ln(n)  +  L16  +  nln  (3  +  7)  +  ln(1  +  R)  ' 


Proof:  We  prove  a  slightly  more  general  result  which  will  imply  the  conclusion  of  Lemma  2.1  as  a 
special  case.  Consider  the  slightly  more  general  definition  of  Wx  parameterized  by  a  scalar  c  >  0: 

Iff  :=  {(w,0)  £  3T+1  :  c-  (w-x)  £  0(S-x),  6  >  0}  ,  (8) 

and  note  that  we  recover  Wx  by  setting  c  =  1.  Notice  that  Iff  n  {(w,  9)  :  9  =  cj  =  S  x  {c},  i.e., 
the  restriction  of  Iff  to  the  slice  of  (x,  9)  £  Kn+1  defined  by  9  =  c  is  simply  S.  We  will  prove: 


In 


(  vol  (Bx+1)  \ 

\vol  (Bx+1  n  Wf)  J 


<  -  ln(n)  +  1.16  +  nln 


(9) 


Notice  that  Lemma  2.1  follows  immediately  from  (9)  by  setting  c  =  1  and  using  the  inequality 

VR2  +  1  <R+  1. 

We  now  prove  (9).  By  hypothesis  there  exists  y  for  which  B(y ,  r)  C  SnB(x,R).  By  performing 
a  translation  if  necessary,  we  can  assume  that  x  =  0,  which  simplifies  the  arithmetic  manipulations 
below.  Define  H  :=  Iff  n  Bf+X  =  {(w,9)  :  c-w/9  £  S,  9  >  0,  |j (it;,  (9) || 2  <  1}  and  T  :=  {(w,9)  : 
c-  w/9  £  B(y,r),  0  <  9  <  c}.  Defining  6  :=  \J R2  +  c2,  we  first  prove  that  T  C  B((0, 0),  5).  To  see 


why  this  is  true,  let  (w,  9)  G  T,  then 


110,60112  =  \\(w  -  (9/c)y+ (6/c)y,0)\\2 


=  || (w  ~  ( 0/c)y  +  (6/c)y\\l  +  92 

<  V(\\(^-(o/Oyh  +  \\(e/OyhV  +  e^ 

<  y/Wc)r  +  (0/c)\\yh)*  +  e* 


<  (. e/c)VR2  +  c 2 

<  <5  . 


Here  the  second  inequality  follows  since  \\(c/6)w  —  y\\  <  r  for  (w,  9)  G  T,  the  third  inequality  follows 
since  B(y,  r)  C  B( 0,  R),  and  the  last  inequality  follows  since  6  <  c  for  (w,  9)  G  T.  This  shows  that 
T  C  R((0, 0),  <5).  Therefore  <5 _1T  C  R((0,0),1)  and  hence  5~lT  C  H.  It  then  follows  that 


Vol (H)  >  Vo 


/  v{n)rn  \  f  9n+1  \  ° 
\5n+1cn )  yn  +  l)  Q 


v(n)rnc 
5n+1(n  +  1)  ’ 


where  v(n)  is  the  volume  of  the  n-dinrensional  unit  ball,  see  (4).  Therefore 


vol  (R*+1)  vol  {Bxn+l)  v(n  +  l)5n+1(n  +  1)  T(n/2  +  l)TT^n+1^25n+1{n  +  1) 

vol  (R*+1  n  W?)  vol  (H)  ~  v(n)rnc  T({n  +  1) /2  +  l)'nn/2rnc 

We  bound  the  right-most  term  involving  the  ratio  of  two  gamma  function  values  using  the  following 
inequality  for  the  gamma  function,  see  [16]: 

r(a+l/2)  ^  1 

r(x  +  i)  <  y/x  + 1/4  ' 

Using  x  =  (n  +  l)/2  in  the  above  yields 

vol  (B*+1)  ^  (  1  ^  7r(n+l)/2<5n+l(w+1) 

vol  (B*+1  n  W*)  —  y  /n+i  I  1  /  7rn/2rnc 


2(n  +  l)y/7r  ^  \/R2  +  c2  ^  ^ \/R2  +  c2^ 

\/2n  +  3  \  r  /  \  c  ) 


< 


3.18  • 


y/n 


VR2  +  C2 
r 
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where  the  last  inequality  follows  from  the  observation  that  —  <  4/^/5  for  n  >  1  since  the 

^/n(2n+3) 

left  expression  is  decreasing  in  n  for  n  >  1.  Lastly,  taking  logarithms  yields  (9).| 


We  acknowledge  C.  Roos  [17]  for  several  constructions  used  in  the  above  proof.  Lemma  2.1  yields 
the  following  complexity  bound  for  solving  (1)  using  the  extended  ellipsoid  method: 


Theorem  2.2  Suppose  that  S  is  given  via  a  separation  oracle,  that  x  £  is  given,  and  that 
the  extended  ellipsoid  algorithm  is  applied  to  solve  (1).  If  R  and  r  are  positive  scalars  such  that 
B(x,  R)  C\  S  contains  a  ball  of  radius  r,  then  the  algorithm  will  solve  (6),  and  hence  solve  (1),  in  at 
most 

2 (n  +  1)  ln(n)  +  1.16  +  nln  (-  +  —  ^  +  ln(l  +  R) 

\2  \r  r J 

iterations,  where  each  iteration  makes  one  call  of  the  separation  oracle  for  S. 


Proof:  The  iteration  bound  follows  immediately  from  Lemma  2.1  and  Theorem  2.1,  noting  that 
the  dimension  of  the  space  containing  Wx  is  n  +  1  | 

Notice  that  this  complexity  bound  requires  only  the  specification  of  an  initializing  point  x, 
which  one  can  think  of  as  the  “reference  point.”  The  condition  “there  exists  R  and  r  for  which 
B(x,R)  n  S  contains  a  ball  of  radius  r”  is  only  existential;  prior  knowledge  of  any  R,  r,  or  bounds 
thereon  are  unnecessary.  The  complexity  bound  is  monotone  in  three  quantities,  namely  R/r,  R, 
and  1/r.  While  it  is  tempting  to  think  that  R/r  will  be  the  largest  of  these  quantities,  Examples 
1.1  and  1.2  of  Section  1  show  that  R  or  1/r  might  be  the  dominant  quantity.  It  is  also  curious 
that  among  these  three  quantities,  the  complexity  bound  depends  more  weakly  on  R  than  on  the 
other  two  quantities,  roughly  by  a  factor  of  n.  We  show  in  Section  3  that  the  dependence  of  the 
complexity  bound  on  R  and  1/r  (as  well  as  on  R/r )  cannot  be  removed.  Therein  we  also  show  a 
weaker  dependence  on  R  than  on  R/r,  but  by  a  different  factor  than  given  above. 

For  a  given  point  x,  we  can  say  that  S  has  favorable  geometry  relative  to  x  to  the  extent  that 
there  exist  values  R  and  r  satisfying  (3)  for  which  R/r,  R,  and  1/r  are  not  too  large.  Put  slightly 
differently,  S  has  favorable  geometry  relative  to  x  if  S  contains  a  ball  whose  radius  r  is  not  too  small 
and  whose  distance  R  from  x  is  not  too  large.  Then  Theorem  2.2  states  that  if  S  has  favorable 
geometry  relative  to  x,  then  the  extended  ellipsoid  algorithm  will  solve  (1)  relatively  quickly.  In 
Section  3  we  study  the  converse  of  this  statement. 

Last  of  all,  notice  that  the  complexity  bound  in  Theorem  2.2  is  not  scale-invariant,  which  seems 
unnatural  at  first  glance.  That  is,  one  would  expect  if  the  units  were  changed  so  that  both  R 
and  r  were  doubled,  say,  then  the  complexity  bound  would  remain  unchanged,  but  this  is  not 
the  case.  The  reason  for  this  has  to  do  with  the  implicit  choice  of  using  c  =  1  in  the  “lift  and 
conify”  construction  used  to  transform  S  to  Wf  =  Wf  =  Wx  in  the  extended  ellipsoid  algorithm. 
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Indeed,  for  a  given  value  of  c  >  0  we  can  implement  the  extended  ellipsoid  method  described  in 
Section  1,  substituting  IV  x  for  Wf  =  Wx.  The  specific  choice  of  c  =  1  used  in  the  description 
of  the  extended  ellipsoid  algorithm  is  somewhat  arbitrary,  but  the  choice  of  c  must  be  given,  i.e. , 
“known”  to  the  algorithm,  so  that  the  separation  oracle  for  S  can  be  converted  to  one  for  W^. 
If  we  change  units  so  that  R  and  r  are  doubled,  then  if  we  double  the  value  of  c  from  c  =  1  to 
c  =  2  it  follows  from  (9)  that  the  volume  ratio  bound  and  hence  the  complexity  bound  will  remain 
unchanged;  hence  the  extended  ellipsoid  method  is  scale-invariant  if  c  is  re-scaled  together  with  R 
and  r.  Furthermore,  it  follows  from  (9)  that  if  a  value  of  R  is  known  in  advance  for  which  (3)  is 
true  for  some  r  >  0,  then  setting  c  =  R  in  (9)  yields  a  volume  ratio  bound  of  0(ln(n)  +  nln(R/r )) 
and  hence  an  iteration  complexity  bound  of  0(nln(n)  +  n2  ln(i?/r)),  whose  sole  dependence  on 
R/r  (and  whose  independence  of  R  and  1/r  separately)  is  consistent  with  the  complexity  bound 
in  Corollary  2.1.  However,  it  is  precisely  because  R  is  not  known  that  the  dependence  on  R  arises 
(even  when  R/r  is  0(1))  in  Lemma  2.1  and  hence  in  the  complexity  bound  of  Theorem  2.2. 

The  ellipsoid  algorithm  belongs  to  a  larger  class  of  efficient  volume-reducing  separation-oracle 
based  algorithms  that  includes  the  method  of  centers  of  gravity  [8] ,  the  method  of  inscribed  ellipsoids 
[7],  and  the  method  of  volumetric  centers  [19],  among  others.  Results  analogous  to  Theorem  2.1 
can  be  derived  for  these  methods,  for  example  for  the  method  of  centers  of  gravity  the  iteration 
complexity  analogous  to  Theorem  2.1  is  0(ln(n)  +  nln(l/r  +  i?/r)-(-ln(l  +  i?)).  For  a  more  thorough 
discussion  of  the  complexity  of  volume-reducing  cutting-plane  methods,  see  [10]. 


3  Lower  Bounds  on  Complexity  of  (1)  for  Separation  Oracle  Al¬ 
gorithms 


Theorem  2.2  showed  that  favorable  geometry  implies  favorable  complexity:  if  R/r,  R,  and  1/r  are 
all  small,  then  the  ellipsoid  algorithm  (and  many  other  algorithms)  will  compute  a  solution  of  (1) 
in  a  small  number  of  iterations.  In  this  subsection  we  study  the  converse  question:  does  favorable 
complexity  imply  favorable  geometry?  A  naive  approach  to  this  question  is  as  follows:  supposing 
that  a  separation-oracle  based  algorithm  solves  (1)  in  a  low  number  of  iterations,  then  prove  that 
R/r,  R,  and  1/r  all  must  be  small.  This  approach  is  obviously  doomed  to  failure,  as  the  algorithm 
could  simply  get  lucky  and  compute  a  solution  of  (1)  by  accident  at  an  early  iteration,  regardless  of 
the  values  of  R/r,  R,  and  1/r.  We  therefore  consider  our  algorithm  applied  not  to  a  single  instance 
of  (1)  but  rather  applied  to  all  instances  in  certain  collections  of  convex  sets.  For  fixed  values  of 
x,  R,  and  r,  let  Sx,r,R  denote  the  collection  of  convex  sets  S  C  whose  intersection  with  B(x,  R) 
contains  a  ball  of  radius  r,  namely: 

gx,r,R  =  /s  C  :  S  is  convex  and  satisfies  (3)  }  . 
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Now  consider  a  (separation-oracle  based)  algorithm  J\4  for  solving  instances  of  (1).  (For  a  precise 
definition  of  a  separation-oracle  based  algorithm  see  [12] ;  an  intuitive  notion  of  this  type  of  algorithm 
is  sufficient  for  our  purposes.)  An  instance  is  a  given  convex  set  S,  or  more  precisely,  a  separation 
oracle  for  the  convex  set  S.  Now  suppose  we  fix  x,  R,  and  r,  and  restrict  our  instances  to  (separation 
oracles  for)  sets  S  in  the  collection  Sx,r,R.  Let  Nx,r,R(A4)  denote  the  computational  complexity  of 
algorithm  A4  over  all  instances  S  £  Sx,r,R.  That  is,  Nx,r’R(A4)  is  the  maximum  number  of  oracle 
calls  it  takes  the  algorithm  A4  to  solve  (1)  over  all  (separation  oracles  for)  sets  S  in  the  collection 
Sx,r,R.  Our  first  lower  bound  result  is  as  follows: 


Theorem  3.1  For  any  fixed  x,  R,  and  r  satisfying  R  >  r  >  0,  let  M.  be  any  separation- oracle 
based  algorithm  applied  over  the  collection  of  sets  Sx,r,R.  Then 


Nx’r’R(M)  > 


Proof:  We  use  a  type  of  argument  used  extensively  in  [12]  that  works  by  constructing  output 
of  a  separation  oracle  for  a  particular  set  S  £  Sx,r,R  for  which  the  algorithm  makes  at  least 
log2  (^R^j  —  lj  oracle  calls.  Without  loss  of  generality  we  presume  that  x  =  0,  which  will  lead 
to  simpler  arithmetic  in  the  proof.  Let  xl  be  the  first  point  used  to  query  the  separation  oracle. 
(This  point  is  generated  by  the  algorithm  independent  of  any  information  from  the  separation 
oracle  or,  equivalently,  the  set  in  question.)  Let  Lq  =  —R  and  Uq  =  R.  If  (xx)i  <  L°+u°  f  the 
oracle  will  return  “x1  ^  S”  together  with  the  separating  halfspace  H++  :=  {x  £  :  x\  >  Lo±ha  j 

for  which  S  C  H++ .  Henceforth  in  this  proof  and  other  proofs  we  simply  denote  this  as  “S  C 
{x  £  :  xi  >  L<i±Lo}.”  jf  instead  (x1)!  >  L°+u° ,  the  oracle  will  return  “x1  ^  S'”  together  with 

“S  C  {x  £  :  xi  <  Lo^Uo}.”  In  the  first  case  we  will  define  L\  :=  L°+u°  and  U\  :=  Uq,  whereas 

in  the  second  case  we  define  L\  :=  Lq  and  U\  :=  Lq+Uq  _  yye  wj}i  construct  the  output  of  the 
separation  oracle  in  subsequent  iterations  in  a  manner  that  generalizes  the  above  logic.  After  k 
oracle  calls  we  will  have  two  scalar  values  Lk  and  Uk  satisfying  L <  Uk,  and  the  algorithm  will 
have  generated  x1, . . .  ,xk  for  which  the  oracle  has  responded  “x*  ^  S”  together  with  separating 
halfspaces  of  the  form  “S  C  {x  £  :  xi  >  (or  <)  L»-i-fW-i  j”  (depending  on  the  position 

of  (x*)i)  for  i  =  1, . . .,  k.  The  algorithm  will  next  generate  xk+1  and  query  the  oracle  with  this 
point.  If  (xfc+1)i  <  LkfUk  ;  the  oracle  will  return  “xfc+1  ^  S”  together  with  the  separating  halfspace 
“S  C  {x  £  :  Xfc+i  >  Lk+Uk If  instead  (xfc+1)i  >  Lk+Uk ;  the  oracle  will  return  “xfc+1  ^  S" 

together  with  the  separating  halfspace  “S  C  {x  £  lRn  :  x\  <  Lk~^Uk}.”  In  the  first  case  we  will 
define  Lf~+ 1  :=  Lk+Uk  and  ■=  Uk,  whereas  in  the  second  case  we  define  Lk+i  ■=  L k  and 

Uk+i  '■=  Lk^Uk .  We  proceed  inductively  until  the  algorithm  has  made  K  =  [log 2(R/r)  —  lj  oracle 
calls  (iterations)  and  we  have  generated  a  (monotone  increasing)  sequence  {Lj}|L0  and  a  (monotone 
decreasing)  sequence  {Ui]f=0  according  to  the  above  rules. 
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Now  define 


S  =  {i  6  F  :  ||x||  <  R,  Lk  +  S  <  x±  <  Uk  —  <5} 

where  5  :=  R-  2~^K+1\  Then  it  follows  that  the  separating  hyperplanes  generated  from  the  oracle 
calls  are  consistent  with  the  instance  S.  We  first  argue  that  the  points  x1, ...  ,xK  ^  S.  To  see  this 
define  Cl  :=  {x  G  :  ||x||  <  R  and  Li  +  <5  <  x\  <  Ui  —  5}  for  i  =  1, . . . ,  K.  Then  it  follows  that 
xl  £  Cl  and  S  C  CK  C  •  •  •  C  C2  C  C1,  therefore  xl  £  S  for  i  =  1, . . . ,  K.  We  claim  that  S  G  Sx,r,R. 
Notice  trivially  that  S  C  B(x,R )  (recall  that  x  =  0).  so  it  remains  to  prove  that  S  contains  a  ball 
of  radius  r.  To  see  this,  notice  that  Ut  —  Li  =  2R2~l  for  all  i  =  0, . . . ,  K,  and  observe  that  the 
upper/lower  bounds  on  the  first  coordinate  of  points  in  S  satisfy: 

UK  -5  -  {Lk  +  <?)  =  UK  -  LK  _  2-(k+i)  =  R  .  2~(k)  _  R  .  2-(k+i)  =  R  .  2-(k+ i)  > 

2  2  “  ’ 

from  which  it  easily  follows  that  B(y,  r)  C  S  for  y  =  {Lk+J±k  0, . . . ,  0).  Hence  S  G  Sx'r,R  and  by 
construction,  x1  (f  S,  i  =  1, . . . ,  K.  Therefore  the  algorithm  makes  at  least  K  oracle  calls,  proving 
the  result.  | 

The  general  technique  used  in  the  above  proof  (and  which  will  also  be  used  to  prove  the  other  two 
lower  bound  theorems  in  this  section)  was  borrowed  from  [12].  Notice  that  it  involves  inductively 
using  the  output  of  the  given  algorithm  Ai  to  create  a  “resisting  (separation)  oracle”  for  a  particular 
(algorithm  dependent)  set  S  G  Sx,r’R,  in  such  a  way  that  the  algorithm  must  make  at  least  a  certain 
number  of  oracle  calls.  (The  appelation  “resisting  oracle”  was  aptly  introduced  by  Nesterov  in  [13].) 
In  the  above  proof,  which  is  essentially  unidimensional,  the  resisting  oracle  is  constructed  in  such  a 
way  that  for  at  least  K  oracle  calls,  the  algorithm  A4  generates  points  xl  which  are  not  contained 
in  the  instance  S.  This  approach  will  be  modified  in  the  proofs  of  the  two  additional  lower  bounds 
of  Theorems  3.2  and  3.3. 

The  method  of  centers  of  gravity  [8]  can  be  shown  to  achieve  the  lower  bound  in  Theorem  3.1; 
the  proof  of  this  result  can  be  gleaned  from  similar  arguments  in  [12] . 

We  next  show  that  the  lower  bound  complexity  also  depends  monotonically  on  R  even  when 
R/r  is  bounded  above  by  a  constant.  Analogous  to  the  family  of  sets  Sx,r,R,  we  will  need  a  suitable 
family  of  (separation  oracles  for)  convex  sets  that  will  serve  as  instances  for  applying  any  algorithm 
A4  to  solve  (1).  For  fixed  values  of  x,  R,  and  aspect  ratio  bound  A,  let  TX,R'A  denote  the  collection 
of  convex  sets  S  C  that  satisfy: 

(i)  there  exists  R  and  f  for  which  B(x,  R)  Pi  S  contains  a  ball  of  radius  f, 

(ii)  R  <  R,  and 

(iii)  R/r  <  A. 

For  a  given  separation-oracle  based  algorithm  M,  let  NX,R'A(. A/f)  denote  the  computational 
complexity  of  algorithm  A4  over  all  instances  S  G  TX’R,A.  That  is,  Nx,R'A(A4)  is  the  maximum 
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number  of  oracle  calls  it  takes  the  algorithm  JA  to  solve  (1)  over  all  (separation  oracles  for)  sets  S 
in  the  collection  TX,R,A.  We  have: 


Theorem  3.2  For  any  fixed  x,  R,  and  A  satisfying  R  >  1  and  A  >  4,  let  JA  be  any  separation- 
oracle  based  algorithm  applied  over  the  collection  of  sets  TX,R,A.  Then 

N*’R’A(M)>[log2log2(R+l)\  . 


We  offer  the  following  interpretation  of  this  theorem  before  proving  it.  For  any  given  algorithm 
M,  there  exists  a  (separation  oracle  for  a)  convex  set  S  £  TX'R,A  for  which  the  iteration  complexity 
of  the  algorithm  grows  at  least  as  log2  log2(i?),  independent  of  1/r  or  the  aspect  ratio  A  of  S, 
provided  that  R  >  1  and  A  >  4. 

Proof:  The  proof  uses  ideas  communicated  privately  from  Nemirovsky  [11],  and  constructs  a 
resisting  oracle  for  a  particular  set  S  £  TX’R'A.  However,  unlike  the  “binary  evasion”  strategy  used 
to  prove  Theorem  3.1,  in  this  proof  the  evasion  takes  place  on  an  exponential  scale,  using  intervals 
of  the  form  [ 2Li ,  2Ui]  as  opposed  to  [Lt.  Uf.  Without  loss  of  generality  we  presume  that  x  =  e1  := 
(1, 0, . . . ,  0),  which  will  lead  to  simpler  arithmetic  in  the  proof.  Let  x1  be  the  first  point  used  to  query 
the  separation  oracle.  (This  point  is  generated  by  the  algorithm  independent  of  any  information 
from  the  separation  oracle  or,  equivalently,  the  set  in  question.)  Let  K  =  [log2log2(l  +  f?)J,  and 
define  Lq  =  0  and  Uq  =  2K .  We  will  construct  the  separation  oracle  inductively  as  follows.  If 

/  1  \  X/0-|-t/Q  i  —  — 

(ar)i  <  2  2  ;  the  oracle  will  return  “x1  ^  S”  together  with  the  separating  halfspace  “S  C  {x  £ 

Lq  +  C/q  ^  _  1  _  Lq  +  Uq  __  __  ,,  i  ,  — 

2  If  instead  (xl)\  >  2  2  ,  the  oracle  will  return  “x1  ^  S”  together  with  the 

° 2  °}.”  In  the  first  case  we  will  define  L\  :=  Lafu° 


:  xi  >  2" 

separating  halfspace  US  C  {x  £  Uin  :  xi  <  2“ 


and  U\  :=  Uo,  whereas  in  the  second  case  we  define  L\  :=  Lq  and  U\  :=  L°+u° .  On  subsequent 
iterations  we  will  construct  the  output  of  the  separation  oracle  in  a  manner  that  generalizes  the 
above  logic.  After  k  oracle  calls  we  will  have  two  scalar  values  L ^  and  satisfying  L ^  <  [/&,  and 
the  algorithm  will  have  generated  x1, . . .  ,xk  for  which  the  oracle  has  responded  “x*  ^  S ”  together 

with  separating  halfspaces  of  the  form  “S  C  {x  £  W‘  :  x\  >  (or  <)  2  2  }”  (depending  on  the 

position  of  (x*)i)  for  i  =  1, . . . ,  k.  The  algorithm  will  next  generate  xk+1  and  query  the  oracle  with 

7  1  7  I  1  — 

this  point.  If  (xfc+i)i  <  2  2  ,  the  oracle  will  return  “xfc+i  ^  S”  together  with  the  separating 

—  Lufi-Uu  .  Lufi-Ub. 

halfspace  “5  C  {x  £  :  Xk+i  >  2  2  If  instead  (xfc+1)i  >  2  5  ,  the  oracle  will  return 

“x/c+i  ^  5”  together  with  the  separating  halfspace  “S'  C  {x  £  :  x\  <  2  2  In  the  first  case 

we  will  define  L^+i  ■=  LkAUk  and  Uk+ i  :=  U^,  whereas  in  the  second  case  we  define  Lfc+i  :=  Ljc  and 
Uk+i  :=  Lk+Uk  ■  We  proceed  iteratively  until  the  algorithm  has  made  K  oracle  calls  (iterations) 
and  we  have  generated  a  (monotone  increasing)  sequence  {L,;}|L 0  and  a  (monotone  decreasing) 
sequence  {Ui}f=Q  according  to  the  above  rules. 
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Now  define  the  following  objects: 


5  :  =  1/4  f  :=  2Ur  —2Lk  _d  R  ._  2uk  _  \ 

y  :=  2Lr  2 21 K  e 1  S  :=  {i  6  P  :  ||x  —  y\\  <  ?}  . 

Then  it  follows  that  the  separating  hyperplanes  that  were  the  output  after  each  oracle  are  consistent 
with  the  instance  S.  We  first  argue  that  the  points  x1, ,  xK  ^  S.  To  see  this  define  Cl  :=  {.x  G 
sftn  .  2 Li  +  5  <  x\  <  2Ui  —  5}  for  i  =  1, ...  ,K.  Then  it  follows  that  xl  ^  Cl  and  S  C  CK  C  •  •  •  C 
C2  C  C1,  therefore  x1  ^  S  for  i  =  1, . . . ,  K . 

We  claim  that  S  G  TX,R'A.  We  first  prove  that  S  C  B(x,  R)  C  B(x,  R).  Indeed,  let  x  G  S,  then 
noticing  that  Ujt  —  Li  =  2K~i  for  all  *  =  0, ,  K  whereby  Uk  —  Lk  =  1,  and  we  have: 

2Lr  -l  2Rk  -  tt  k 

\\x— x\\  <  ||a;— y||  +  ||y— x\\  <  r-\ - - - 1  =  2Uk—8—\  <  R  <  2Uo—l  =  22  — 1  <  R+ 1—1  =  R  , 


thus  showing  that  x  G  B{x ,  R)  C  B(x,  R)  and  proving  this  first  part  of  the  claim. 
We  complete  the  proof  by  showing  that  R/f  <  A.  First  observe: 


2Ur  _  21 


)Lk+ i 


r  = 


-5  = 


-  21 


2  2 
from  which  it  follows  that: 

R  2Uk  —  1 


-<5  = 


2l° 


1 


--l/4>--l/4=--i  =  l/4, 


2lk+i  _  ^  2Liv'+1  —  1 


2uk  —2lk  $  2 lk-1  -  S  2Lk~1  —  1/4 


=  4  <  A 


Hence  S  G  TX’R’A  and  by  construction,  xi  (j  S,  i  =  1, . . . ,  K.  Therefore  the  algorithm  makes  at 
least  K  oracle  calls,  proving  the  result.  | 


Remark  3.1  Note  in  the  proof  of  Theorem  3.2  that  the  instance  S  constructed  in  the  proof  has 
an  aspect  ratio  bounded  above  by  4,  and  also  satisfies  r  >  1/4.  Therefore  we  could  equivalently 
re-phrase  Theorem  3.2  to  state  that  A  =  4  rather  than  A  >  4. 


Last  of  all,  we  show  that  the  lower  bound  complexity  also  depends  monotonically  on  1/r  even 
when  R./r  is  small.  Analogous  to  the  family  of  sets  TS,R,A ,  we  will  need  a  suitable  family  of 
(separation  oracles  for)  convex  sets  that  will  serve  as  instances  for  applying  any  algorithm  A4  to 
solve  (1).  For  fixed  values  of  x,  r,  and  aspect  ratio  bound  A,  let  Ux,r,A  denote  the  collection  of 
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convex  sets  ScJf"  that  satisfy: 

(i)  there  exists  R  and  f  for  which  B(x,  R)  n  S  contains  a  ball  of  radius  f, 

(ii)  r  >  r,  and 

(iii)  R/r  <  A. 

For  a  given  separation-oracle  based  algorithm  At,  let  Nx,r,A(Al)  denote  the  computational 
complexity  of  algorithm  At  over  all  instances  S  €  Ux,r,A.  That  is,  Nx,r-A  (A4)  is  the  maximum 
number  of  oracle  calls  it  takes  the  algorithm  At  to  solve  (1)  over  all  (separation  oracles  for)  sets  S 
in  the  collection  Ux,r,A.  We  have: 


Theorem  3.3  For  any  fixed  x,  r,  and  A  satisfying  0  <  r  <  1/4,  and  A  >  4,  let  At  be  any 
separation- oracle  based  algorithm  applied  over  the  collection  of  sets  Ux,r,A .  Then 


Nx’r'A(M )  > 


log2 log2 


We  offer  the  following  interpretation  of  this  theorem  before  proving  it.  For  any  given  algorithm 
At,  there  exists  a  (separation  oracle  for  a)  convex  set  S  6  Ux,r’A  for  which  the  iteration  complexity 
of  the  algorithm  grows  at  least  as  log2  log2(l/r),  independent  of  R  or  the  aspect  ratio  A  of  S, 
provided  that  r  <  1/4  and  A  >  4. 


Proof:  The  proof  is  similar  in  structure  to  that  of  Theorem  3.2,  except  that  instead  of  exponentially 
scaled  intervals  [ 2Li ,  2Ui]  we  use  inverse  exponentially  scaled  intervals  [2~Li,  and  some  other 

arithmetic  is  different  as  well.  Let  K  =  [log2  log2(l/(4r))J ,  Lq  :=  2K ,  Uq  :=  0,  and  without  loss  of 
generality  we  presume  that  x  =  (l/2)i°e1,  which  will  lead  to  simpler  arithmetic  in  the  proof. 


Let  x1  be  the  first  point  used  to  query  the  separation  oracle.  (This  point  is  generated  by  the 
algorithm  independent  of  any  information  from  the  separation  oracle  or,  equivalently,  the  set  in 
question.)  We  will  construct  the  separation  oracle  inductively  as  follows.  If  (ar)i  <  (1/2)  2  , 

the  oracle  will  return  “x1  (/  S”  together  with  the  separating  halfspace  “S  C  {x  €  $tn  :  x\  > 

Lq-\-Uq  _ _  ^  Lq-\-Uq  __  _  ,,  1  ,  — 

2  If  instead  (a;  )i  >  (1/2)  2  ,  the  oracle  will  return  “ar  ^  5”  together  with  the 


L0  +  u0  - 


_ Lq+Uq 


(1/2)' 

separating  halfspace  “5  C  {x  €  ffl1  :  x\  <  (1/2)  h  In  the  first  case  we  will  define  L\  := 
and  U\  :=  Uq,  whereas  in  the  second  case  we  define  L\  :=  Lq  and  U\  :=  L°fu° .  On  subsequent 
iterations  we  will  construct  the  output  of  the  separation  oracle  in  a  manner  that  generalizes  the 
above  logic.  After  k  oracle  calls  we  will  have  two  scalar  values  L *.  and  satisfying  L *.  >  Uk,  and 
the  algorithm  will  have  generated  x1, . . .  ,xk  for  which  the  oracle  has  responded  “x*  ^  S”  together 

with  separating  halfspaces  of  the  form  US  C  {x  €  :  x\  >  (or  <)  2  2  }”  (depending  on  the 

position  of  (x*)i)  for  i  =  1, . . . ,  k.  The  algorithm  will  next  generate  xk+1  and  query  the  oracle  with 

7  1  7  1  1  — 

this  point.  If  (xfc+i)i  <  (1/2)  2  5  the  oracle  will  return  “xfc+i  ^  S”  together  with  the  separating 
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halfspace  “S  C  {i  £  !!“  :  Xk+ 1  >  (1/2)  k 2  ^  }.”  If  instead  (xfc+1)i  >  (1/2)  l2  fe,  the  oracle  will 

return  “xr+i  (/  S”  together  with  the  separating  halfspace  “S  C  {i  6  If  :  ii  <  (1/2)  5 

In  the  first  case  we  will  define  L^+i  :=  Lk+Uk  and  t/fc+i  :=  U^,  whereas  in  the  second  case  we 
define  Lk+\  :=  L and  U^+i  '■=  LkAUk  .  We  proceed  iteratively  until  the  algorithm  has  made  K 
oracle  calls  (iterations)  and  we  have  generated  a  (monotone  decreasing)  sequence  0  and  a 

(monotone  increasing)  sequence  {C/j}/h0  according  to  the  above  rules. 

Now  define  the  following  objects: 


5  :=  min  \  |  ( 1 


_  .=  (1/2)uK-(1/2)lk  5 


R  :  =  (1/2)^  -  (l/2)io 


y  ■ 


:=  d/2)^+(i/2 )^ei  5  ,=  {x  g  3fJn  ;  |jx  —  y||  <  r}  . 


Then  it  follows  that  the  separating  hyperplanes  that  were  the  output  after  each  oracle  are 
consistent  with  the  instance  S.  Note  that  R  >  0;  to  see  this  observe  that  Li  —  Ui  =  2K~i  for  all 
i  =  0, . . . ,  K ,  whereby  Lk  —  Uk  =  1  and: 


*=<i 


l\L° 

V 


'  1  \  Lo-Lk\ 
2-1-  >0 


We  first  argue  that  the  points  x1, . . .  ,xK  S.  To  see  this  define  Cn  :=  {x  G  :  (l/2)Li  +  d  < 
x'i  <  (\/2)Ui  —  $}  for  i  =  1, . . . ,  K.  Then  it  follows  that  xl  (/  Cn  and  S  C  CK  C  •  •  •  C  C2  C  C1, 
therefore  xl  ^  S  for  i  =  1, ...  ,  K. 

We  claim  that  S  G  Ux,r,A.  We  first  prove  that  S  C  B(x,  R).  Indeed,  let  x  G  S,  then  we  have: 
\\x  -  x||  <  \\x  —  y\\  +  \\y-x\\  <  f  +  (V2)Lk  +  (V2)^  _  (1/2)l0  =  (i/2)^  -  (l/2)L°  -5  <  i?  , 

thus  showing  that  x  G  B{x,  R)  and  proving  this  first  part  of  the  claim. 

We  next  show  that  f  >  r.  We  have: 

^  (t/2)^-1  -  (1/2)L*  1  / 1  \  1  /1\L°  1/1 

r_  2  ~~  2  V2/  “  2  V2/ 

Last  of  all,  we  show  that  R/r  <  A.  which  will  complete  the  proof.  We  have: 

R  (1/2)^  -  (1/2)L°  2(1/2)^  -  (1/2)l° 


i  nK  i  ~  5  >  —  ~  5  =  2r  —  5  >  r 
2  \22K)  ~  2 


m 


< 


-5 


mK-m 


Lq 


=  4  <  A 
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Hence  S  G  ly(x’r,A  and  by  construction,  xl  ^  S,  i  =  1  Therefore  the  algorithm  makes  at 

least  K  oracle  calls,  proving  the  result.  | 

Remark  3.2  Note,  just  as  in  the  case  of  Theorem  3.2,  that  in  the  proof  of  Theorem  3.3  that  the 
instance  S  constructed  in  the  proof  has  an  aspect  ratio  bounded  above  by  4.  (Also  notice  in  the 
proof  that  R  <  1.)  Therefore  we  could  equivalently  re-phrase  the  theorem  to  state  that  A  =  4  rather 
than  A  >  4. 


4  Discussion  and  Further  Questions 

4.1  On  Stronger  Lower  Bounds 

There  is  a  significant  gap  between  the  lower  bound  results  in  Theorems  3.1,  3.2,  and  3.3  and 
the  upper  bound  result  in  Theorem  2.2.  One  reason  for  the  gap  has  to  do  with  the  absence  of 
any  dimensional  factor  n  in  the  lower  bound  results.  This  is  partially  an  artifact  of  the  proof 
constructions  in  the  lower  bound  theorems,  which  are  all  essentially  unidimensional  in  nature. 
It  might  be  of  interest  to  strengthen  the  lower  bound  theorems  by  taking  explicit  advantage  of 
the  dimension  n  in  constructing  suitable  resisting  oracles  with  stronger  lower  bounds.  But  even 
when  n  =  1  there  is  a  gap  between  the  lower  bound  and  the  upper  bound  results  concerning  the 
dependence  on  R  and  1/r  in  Theorems  3.2  and  3.3,  by  a  logarithmic  factor.  It  would  also  be 
interesting  to  strengthen  these  lower  bound  results  by  removing  the  extra  logarithmic  factor  in  the 
lower  bound  results. 


4.2  Lower  Bounds  on  Complexity  for  the  (self-concordant)  Barrier  Model 

Taken  together,  Theorems  3.1,  3.2,  and  3.3  show  that  there  exist  problem  instances  in  certain  fam¬ 
ilies  characterized  by  unfavorable  geometry,  that  require  more  computational  effort  to  solve  by  any 
separation  oracle  algorithm.  This  naturally  begs  the  question  whether  such  an  implication  might 
extend  to  the  case  where  the  set  is  described  instead  by  a  self-concordant  barrier.  In  particular, 
consider  the  case  when  S  is  presented  in  conic  format  (2)  and  the  cone  K  has  a  computable  self- 
concordant  barrier.  It  is  shown  in  [5]  that  in  this  case  favorable  geometry  of  S  implies  favorable 
computational  complexity  of  a  suitably  defined  barrier  method  for  solving  (1).  However,  it  is  an 
open  challenge  to  prove  an  assertion  that  unfavorable  geometry  implies  (say,  in  the  worst  case)  a 
large  number  of  iterations  of  any  (self-concordant)  barrier  method. 
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